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Abstract— We solve the correspondence problem in active stereo 
vision using a novel pseudorandom coded structured light (SL). This 
coding scheme performs well in the presence of occlusion. In settings 
where color coding is feasible. 3D information can be obtained usina a 
single image. 
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1 Introduction 

1.1 Overview 

One of the primary aims of machine vision for the past twenty 
years has been to find a method for rapidly acquiring the three 
dimensional coordinates of points in a scene. Typically, images of 
the scene are obtained from two distinct vantage points. Points in 
the scene- are then located in each image and a trianguiation is 
performed to find the 3D (or world) coordinates of these points [3], 
(4), [14]. The task of identifying the same point in the two images is 
known as the correspondence problem. 

There has been substantial effort put into solving the corre- 
spondence problem. Some of the most successful approaches 
utilize structured light (SL). In this technique, one camera in the 
stereoscopic pair of cameras is replaced by a projector which 
illuminates the scene with a pattern of light [9], [10]. This re- 
duces the correspondence problem to matching illuminated 
scene points on a single image with those on the projection 
plane. Design of an effective illumination pattern is kev to sim- 
plifying this indexing task and a number of approaches have 
been developed. These approaches fall into two general classes- 
temporal codes, where each point is identified bv a unique time- 
modulation of its intensity [1], (2); and spatial codes, where the 
position of a point relative to a set of distinct markings uniquelv 
defines its identity [6|. [7), [16]. The coding methods being intro- 
duced in this paper can be used in temporal as well as spatial 
implementations of SL. 

2 The PSM Codes 

We present here an alternative SL code based on pseudorandom 
arrays. A pseudorandom array is an A-ary array of size k x I 'm 
which many of the possible .4-ary matrices of size ofvXw (of 
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which there are a total of A™) appear exactly once as windows 
(8). Clearly, such a structure allows complete global position 
information of an element based only on the v.xw window sur- 
rounding it. Note that pseudorandom arrays are related to per- 
fect maps, which are simply arrays in which every possible v X :v 
window appears exactly once [18]. Because of this, we also refer 
to our arrays as perfect submaps (PSM). 

2.1 Properties of the Code 

The code we have developed consists of a square pseudorandom 
matrix. Ml of size L (k = / = L). Each element m- within M is as- 
signed one letter from a palette of A possible letters. Since M is a 
pseudorandom array, the location in M of each element m, (ex- 
cept those along the border) is completely determined by the 
letters contained in the v x w neighborhood of m tj . In this case, 

the position of each m tj is indexed bv a nine-element vector V, 
where, "' 

^3/i = K,)i ; v (//i2 ; v (//}3 ; v (//h ; v ,/, )5 ; v ( ,» ; v (l7|7 ; v (v)8 ; v ( ,;)4 

Assignment of letters to the matrix elements is controlled so 
that each V iij} differs from all other V irf , The separation between 
each pair must equal or exceed a specified minimum Hamming 

distance, h, where the Hamming distance between V m and V frn is 
given by, 



"(9-«7) = where 6, = l° * V 



1 otherwise 

Higher Hamming distance codes can be used to perform error 
detection and correction via standard techniques of communica- 
tion theory [15]. The feasibility of constructing a pattern where 
minimum H(ij, i'j') z U (for 1 < /, j < L, and (i. j) * (f , /')) depends 
on L, A, h, and it'. To simplify our discussion, we are limiting our 
examples to 3 x 3 window sizes and a Hamming distance of one or 
three (i.e., w = 3, and //=lor/i = 3), however our approach applies 
equally well to other window sizes and Hamming distances. 

Different elements of A in M are identified at the camera side 
by giving them different attributes on the projection side. These 
can be color, shape, temporal encoding, etc. It is therefore desir- 
able to use the smallest possible value of A. For example, in a 
color encoded perfect submap using fewer colors will simplify 
the determination of a given point's color in a scene with com- 
plex background and surface reflectivity; in a temporal encoded 
implementation it will decrease the acquisition time. We know 
that there are only A* distinct V iij} (code words), and that (L - 2) : 
words are necessary to construct M. Therefore, a lower bound on 
A can be defined as A > (L - 2) a/H) . Unfortunately, a direct solu- 
tion of M for an arbitrary set of A, h. xv is not possible. We em- 
ployed a pseudorandom method of generating M that uses a fill- 
and-test-as-you-go-along construction scheme, which is ex- 
plained next. We note here that for certain values of L A, and xv 
with h - \ t there exist construction methods; the constructed 
arrays are usually nonsquare [8], (12], [13], [15], 

2.2 Pseudorandom Code Generation 

We begin the pattern generation process by seeding the top left 3x3 
window of the matrix with random letter assignments (Fig. 1). The 
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(a) 



Fig. 1. Code generation. The matrix elements are represented here bv 

I 3 " 6 ! (b,aCk d0t ' letter 1: 9ray dot ' letter ^ hatched dot. letter 3 
and white dot, unassigned). 

codes are generated by iterating between adding one new code 
word and checking it against all preceding ones. Pattern growth at 
the top and left margins of the matrix consists of a random as- 
signment of letters from the available palette for each of the three 
positions adjoining the established pattern (Fig. 1 top right and 
bottom left). This action generates one new V aj , For the rest of the 
matrix, a new V (fj) is generated by the addition of a single point to 
the pattern (Fig. 1 bottom right). In ail cases, the Hamming dis- 
tance between the new code word and all of those previously 
embedded in the matrix is computed. If all of these distances are 
equal to or greater than the preset rninimurn Hamming distance ft 
the new code is accepted and the algorithm proceeds with the next 
iteration. If not, a new letter assignment is generated. If, in a par- 
ticular iteration, all possible letter assignments are exhausted (ie 
none were found to have a sufficient Hamming distance), the ma- 
trix is cleared and the process started over from the beginning 

Using the above method, 1,000 trials were executed in an at- 
tempt to generate a 45 x 45 size arrav for various values of h 4 
and uk The values tested were all combinations of A, w and ft 
where A . 1 to 4M - 3 to 9; „ . 3 to 5. The search results for all of 
these cases are given in Table 1 in the Appendix. With window size 
restricted to w = 3, we required A > 8 to achieve h > 3, while to 
achieve h > 1 we required A > 3. Although not shown in the table 
it was possible to obtain a minimum Hamming distance of five! 
but only with to = 5 and A = 6. 

2.3 Correspondence Assignment in PSM 

Our SL system consists of a CCD camera and an LCD or a slide 
projector. The projector is used to illuminate the object with a 
two dimensional, PSM encoded dot array. The camera located at 
a second vantage point captures the scene and digitized images 
are ted to a computer. Fig. 2a shows an experiment where a sfep 
« put in the scene which hides part of the projected pattern from 
the camera. SL using a projected grid with a fixed sampling fre- 

S%i may f3il I" Sethng fFig - 2b) - Slnce P" { e« submap 
based SL uses only local information this hidden step does not 

substantially effect performance. This capability comes from the 
coding properties of the projected pattern as discussed next 

Analysis starts with extraction of the coordinates of the dots' 
centroia^ from the captured images. For each dot, the eight clos- 
est neighbors are found, forming a 3 x 3 window. Each dot also 
carries a property, U ke a color or a temporal on-off pattern which 




Hidden step line 



(b) 

Rg. 2. (a) Structured light setup and an example of occlusion. Part of 
he scene illuminated with the SL pattern is not visible to the camera In 
this image a diagonal step (alpha = 45°) is present in the scene, (b) 
Structured light using grid illumination. Using a grid on the setup shown 
in (a) may not detect the occlusion step. 

gives it a letter assignment. As a result, each window generates a 
rune-letter code word which can be used to index its projection 
coordtnates. This correspondence process varies slightly when 
different Hamming distance codes are used. 

In a Hamming distance one map, once a 3 x 3 window is estab- 
lished and its location in the projected code is found, all of its nine 
elements can be tentatively assigned a correspondence. Each dot 
will be assigned nine times since it is a member of nine different 
but overlapping 3 x 3 windows. These nine assignments can be 
corroborative or contradictory The extent of agreement for a par- 
ticular match defines the confidence level of that correspondence. 
Typically, for a planar object at the middle of the field of view, all 
points are matched with a confidence of nine, while at the sides 
and corners, the confidence levels will be lower (Fig. 3a). 

In the Hamming distance one case, when a point in the center 
of the field of view is not detected, the confidence level of its 
neighbors decreases, but the effect remains localized (Fig. 3b). If 
that point is detected but its property (e.g., color) is misidenti- 
hed, every window containing that false letter will be misla- 
beled. Mislabeling of a window results in a false assignment of 
all of its elements. At the end, a given element may receive its 
assignments from both correctly labeled and mislabeled win- 
dows. While the former are corroborative, the latter are not. By 
tabulating the assignments for each dot, the correspondence 
with the highest confidence value is determined. If that value 
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Fig. 3. Correspondence assignments in PSM based SL. (a) Decreased confidence numbers at the edges and comers of a projected 9x9, Ham- 
ming 1 array, (b) Decreased confidence numbers around a missing point in an h = 1 array. The 9x9 arrays in (b) through (f) are taken from a 
center area of a larger projected array, so edge and corner effects are not present, (c) Confidence numbers around a misidentified central point in 
an h = 3 array, (d) Confidence numbers around a perpendicular occlusion, Hamming distance of the array is 1 . (e) Confidence numbers around the 
occlusion placed at 45° (h = 1). (f) Confidene numbers, around the occlusion placed at 60° (or 30°), n= 1. 



exceeds the next highest confidence value by a fixed certainty 
factor then the correspondence with the highest value is as-- 
signed to the dot. If it does not, that dot is left unlabeled. 

In Hamming distance three case, each code word is separated 
by at least three letters from all others. In other words, each code 
in a Hamming distance three map has a unique Hamming dis- 
tance one basin (attractor). This basin allows correct window 
identification even if one of the letters is incorrect or missing. In 
our SL implementations with Hamming distance three PSM 



codes, we label each 3x3 window nine times, each time by dis- 
regarding one of its nine elements. If there are no errors in any of 
the elements, all of them will be assigned a confidence of nine 
from this window's label. If there is a one element error, a win- 
dow containing the erroneous element can be correctly labeled 
only once out of the nine assignments. As before, each point is 
also a member of nine different windows, so in the case of 
Hamming distance three, each point's confidence can be as high 
as 81 (nine different windows, each window assigned nine 
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times). In the case of a misidentified center dot, we get a confi- 
dence of nine for the correct correspondence candidate for that 
dot, since all the nine windows which it is a member of can be 
assigned once correctly (Fig. 3c). The remaining 72 assignments 
will yield low confidence candidates. 

The effects of occlusion on confidence levels can be seen in Figs. 
3d, 3e, and 3f for the case of Hamming distance one. The arrange- 
ment of erroneous elements in windows straddling the occlusion 
depends on the angle of the discontinuity. In Figs. 3d, 3e, and 3f, 
the number of correct windows is shown for every element- This 
also equals the confidence values of correct correspondences. For 
points near the occlusion, there will also be incorrect correspon- 
dence assignments propagated from false window labels. 

Depending on the orientation of the occlusion in relation to 
the projected code, the Hamming properties of the code, and the 
certainty factor, the percentage of successfully matched points 
will differ. These dependencies will be examined in an experi- 
mental setting next. In the real world, missing or misidentified 
points, corner and edge effects and occlusion problems may 
overlap and may result in dots left unlabeled. One has to choose 
the code properties and the certainty level to optimize sensitivity 
and specificity of correspondence, tailoring the SL system to the 
individual needs. 

3 Experimental Results 

To evaluate performance of the PSM-based SL, we imaged the 
scene illustrated in Fig. 2a. It consists of an occlusion step posi- 
tioned at three different angles to the horizontal (alpha = 90°, 60°, 
and 45*). For each angle, part of the projected array on the lower 
plane was occluded from the camera. Three different types of bi- 
nary temporal coding sequences were used in the experiment: 

1) A classical 2D binary sequence (CBS) where no neighbor- 
hood relations are employed and each point is uniquely 
identified by its own temporal on-off pattern of illumina- 
tion, as in Altshuler et al. [1], 

2) Hamming distance one PSM, and 

3) Hamming distance three PSM. 

The images were processed and a correspondence was estab- 
lished for each type of coding sequence at all three angles of the 
step. The correspondence efficiency ratio is defined as, the num- 
ber of sample points correctly identified relative to the number 
of visible projected points. The ratios for 60, 45, and 90 degrees 
of occlusions for Hamming 1 and certainty factor two were 
366/431 (So percent), 373/430 (87 percent), and 387/428 (90 per- 
cent), respectively. The ratios increased when a hamming dis- 
tance three code has been used: 402/431 (93 percent), 417/430 
(97 percent), 421/28 (98 percent). All of the assigned correspon- 
dences were correct. The unassigned points includes points lost 
due the edge and corner effects. 

The PSM based structured light has high efficiency (85 per- 
cent or higher), but not as high as CBS (100 percent). In CBS, 

log : (N) + 1 images have to be captured in order to uniquely 

label N beamlets. In our noncoplanar setting, CBS acquisition 
needed a minimum of 10 slides (nine-bit code for each point 
and, as will be explained below, one sampling frame), while 
imaging with a Hamming 1 PSM can be achieved with three 
slides and a Hamming 3 PSM employs only four temporal 
slides. In each of these cases, one frame, the sampling frame, is 
used in the extraction of centroids. For this frame, all dots are 
illuminated. The use of a sampling frame provides some toler- 
ance for subject movement between the different temporal 
slides. The remaining slides are indexing frames with each 
letter defined by the temporal sequence of illumination. Thus, 
a four-frame temporal code consists of three letter indexing 
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Fig. 4. Color encoded PSM. A hemicylinder was illuminated by slide projec- 
tor from above containing a 35-mm slide with a 45 x 45, Hamming 1 , 
three-color PSM dot array. The view from a color camera mounted 
lateral to the projector is shown in (a) The computed surface (b) is incom- 
plete due to difficulty in color dot segmentation in receding surface. A 
second camera provided a better vantage view of that section of the 
hemicylinder and combination of the points extracted from both views 
and surface fitting yielded amore complete rendition of the object (c). 

frames and one sampling frame, and these three indexing 
frames can accommodate up to 8(= 2 3 ) letters. 

3.1 Color Code Implementation 

An attractive implementation of PSM employs color encoding. 
The number of image frames required can then be reduced fur- 
ther. If a three-chip camera is employed, a single frame may suf- 
fice. We implemented a code generated using the search algo- 
rithm described above (L = 45, w = 3, A = 3, and h = 1) as a red, 
blue, and green dot array on a 35-mm slide. In Fig. 4a, projection 
of this code on a simple geometric object, a hemicylinder, is il- 
lustrated. From such a single frame both indexing information as 
well as centroid locations can be obtained. At the receding top 
edge of the cylinder, labeling efficiency decreases (Fig. 4b). None- 
theless, due to the local nature of PSM coding, this has no global 
consequences. If mapping a more complete surface is desired, 
views from multiple cameras may be readily combined (Fig. 4c). 
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TABLE 1 

The Results of the Pseudorandom Search Algorithm for a Matrix with i = 45 
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A difficulty with color encoding of PSM arises if the imaged 
subject has complex reflectivity properties. Color identification 
may then be compromised. This is particularly severe if more than 
three colors are employed as would be required for codes with 
Hamming distance greater than one. 

4 Summary and Conclusion 

The structured light coding scheme discussed here simplifies the 
identification of points in the illumination pattern (the corre- 
spondence problem). Because it employs only local information, 
the labeling of the projection pattern is less sensitive to missing 
points than other previously described spatial codes such as the 
ones which have used grids [7). Labeling of the grid intersection 
points can be time consuming and complex, especially if parts of 
lines are occluded. Although they only require one image to 
uniquely label many points, it requires a depth first search in 
which each new label is dependent on previously labeled points. 

Our method does not depend on the acquisition of as many 
images as do classical binary temporal codes [1], so it can be 
better suited for applications where movement is a problem and 



a small reduction in the sampled points can be tolerated. Fur- 
thermore, the technique offers the possibility of employing 
neighborhood-level error detection and correction based only on 
the local characteristics of the code, similar to the approach 
taken by Boyer and Kak [5J. Their approach relies on local in- 
formation and attempts to index individual stripes by a "crystal 
growing" process where subpatterns of stripes act as seeds to the 
stripe indexing decision process. Although each subpattern is 
unique and the total code is formed by putting them side by- 
side, the uniqueness of the overlapping codes is not assured. 
Pseudorandom arrays eliminate this problem since all code 
words must be unique. In addition, our two dimensional codes 
allow the flexibility of working in noncoplanar settings and re- 
quire less number of colors (or other attributes) for a given hori- 
zontal resolution. Unlike the crystal growing' method, corre- 
spondence assignments with PSM are accomplished without 
iterating. In the scenes where color imaging is feasible, our tech- 
nique has the potential of capturing the 3D information using 
only a single color frame. 
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APPENDIX 
TABLE 1 

The results of the pseudorandom search algorithm for a matrix 
with L = 45. Three parameters summarize the 1,000 searches for a 
given ft and A: Number completed, Maximum completed and ^Av- 
erage. Number completed gives the number out of WOO tna Is 
which were successful. Maximum completed will be 100 , a tul 
code is generated. Otherwise, it gives the maximum percentage of 
the matrix that was filled by the algorithm. The average of the 
fiUing percentage of the matrix in all of the 1.000 attempt .s given 
in Average. Other tables for L = 23 and L = 35 can be found ,n [11], 
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Multispectral Random Field Models for 
Synthesis and Analysis of Color Images 

Jesse Bennett and Alireza Khotanzad 

Pit — 537- ~(£) 
Abstract-in th'is paper, multispectral extensions to the traditional gray 
fevSunaneousaStoregressive (SAR) and Markov ^om fteW 
MRn models are considered. Furthermore, a new image model is 
S^ tne^eudTMarkov model, which retains the charactenst.es 
I ^S^Markov model, yet admits to a simplified parameter 
es imation method. These models are well-suited to analys.s and 
SSSSXtof images. For each model considered. £™£Z enfal 
developed lor parameter estimation and .mage s V^ e f 
results based on known image models and natural texture samples, 
substantiate the validity of these results. 
Index Terms-Color texture models, color texture synthesis, color 
exiure analysis multispectral random fields, mult.spec.tral 
stm ^regressive models, multispectral Markov rrandom 
field models, multispectral pseudo-Markov random held models, least 
squares estimation. 
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1 Introduction 

THE class of random field (RF) models based on intensity images 
has been widelv used in many application areas, including image 
svnthesis and analysis [1), [2), image compression 1.3], geode S} [4J 
a^d agriculture R W To date, however, there has been h tie work 
in the area of modeling color images usmg these methods. A 
method for image segmentadon based on a color random field 
model can be found in [7], bu, there has been no pubhshed work 
which covers these models in detail. i.e., a complete analytical 
model, devoid of simplifying assumptions, which include s me tiv 
ods for image svnthesis as well as parameter eshmahon. This i the 
poblem addressed in this paper. Applications of this work mdude 
color image synthesis, compression, classification, and ^ enia ^ 
Previous methods for modeling color texture typically involve 
mapping color to gray tones [8], which allows me use of existing 
gray tone models. Although the simplicity of this approach 
Stive, the resulting models suffer considerable loss o mage 
information. In this work, the use of multispectral r-dom fields a 
an image model is considered. In the multispectral case, RF models 
are defined as intensity levels on multiple two dimensional lattice 
planes, with the intensity level at each lattice location taken to be a 
linear combinarton of neighboring intensity levels and an additive 
noise component. For an image with P such intensity ^ planer these 
models are characterized by P" sets of relative neighbor locations 
and neighbor coefficients. 

For mathematical simplicity, the models considered here are 
formulated using the toroidal lattice assumption. A locafon 
within an M x M two dimensional lattice is denoted by s = (s„ 

s ,), with s„ S, integers from the set J = |0, 1 M - 11. The set of 

all lattice locations is defined as O = Is = (s„ > 2 ) : s„ s : € J|. The 
value of an image observation at location s is denoted by the 
vector value y(s), and the image observations are assumed to 
have zero mean. 
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